Using SPAWN and Pipes
IDL’s SPAWN procedure spawns a child process to execute a command or series of commands. General use of SPAWN is described in detail in the IDL Online Help. This section describes how to use SPAWN to communicate with the spawned child process using operating system pipes.
By default, calls to the SPAWN procedure cause the IDL process to wait until the child process has finished before continuing, with output sent to the standard output or captured in an IDL variable. Alternatively, IDL can attach a bidirectional pipe to the standard input and output of the child process, and then continue without waiting for the child process to finish. The pipe created in this manner appears in the IDL process as a normal logical file unit.
Once a process has been started in this way, the normal IDL input/output facilities can be used to communicate with it. The ability to use a child process in this manner allows you to solve specialized problems using other languages and to take advantage of existing programs.
In order to start such a process, use the UNIT keyword to SPAWN to specify a named variable in which the logical file unit number will be stored. Once the child process has done its work, use the FREE_LUN procedure to close the pipe and delete the process.
When using a child process in this manner, it is important to understand the following points:
- Closing the file unit causes the child process to be killed. Therefore, do not close the unit until the child process completes its work.
- A pipe is a buffer maintained by the operating system with an interface that makes it appear as a file to the programs using it. It has a fixed length and can therefore become completely filled. When this happens, the operating system puts the process that is filling the pipe to sleep until the process at the other end consumes the buffered data. The use of a bidirectional pipe can lead to deadlock situations in which both processes are waiting for the other. This can happen if the parent and child processes do not synchronize their reading and writing activities.
- Most C programs use the input/output facilities provided by the Standard C Library (stdio). In situations where IDL and the child process are carrying on a running dialog (as opposed to a single transaction), the normal buffering performed by stdio on the output file can cause communications to hang. We recommend calling the stdio setbuf() function as the first statement of the child program to eliminate such buffering.
(void) setbuf(stdout, (char *) 0);
It is important that this statement occur before any output operation is executed; otherwise, it may not have any effect.
Example: Communicating with a Child Process via an Operating System Pipe
The C program shown in the following example (test_pipe.c) accepts floating- point values from its standard input and returns their average on the standard output. In actual practice, such a trivial program would never be used from IDL, since it is simpler and more efficient to perform the calculation within IDL itself. The example does, however, serve to illustrate a method by which significant programs can be called from IDL.
In the interest of brevity, some error checking that would normally be included in such a program has been omitted. For example, a real program would need to check the non-zero return values from fread(3) and fwrite(3) to ensure that the desired amount of data was actually transferred.
The code for this example can be found in the spawn subdirectory of the external directory of the IDL distribution. Instructions for building it can be found in the README file located in that directory.
This program performs the following steps:
- Reads a long integer that tells how many data points to expect, because it is desirable to be able to average an arbitrary number of points.
- Obtains dynamic memory via the malloc() function, and reads the data into it.
- Calculates the average of the points.
- Returns the answer as a single floating-point value.
Since the amount of input and output for this program is explicitly known and because it reads all of its input at the beginning and writes all of its results at the end, a deadlock situation cannot occur.
The following IDL statements use test_pipe to determine the average of the values 0 to 9:
PRO test_pipe
; Start test_pipe. The use of the NOSHELL keyword is not
; necessary, but serves to speed up the start-up process.
SPAWN, ’test_pipe’, UNIT=UNIT, /NOSHELL
; Send the number of points followed by the actual data.
WRITEU, UNIT, 10L, FINDGEN(10)
; Read the answer.
READU, UNIT, ANSWER
; Announce the result.
PRINT, ’Average = ’, ANSWER
; Close the pipe, delete the child process, and deallocate the
; logical file unit.
FREE_LUN, UNIT
END
Executing the IDL TEST_PIPE procedure gives the result:
Average = 4.50000
This mechanism provides the IDL user a simple and efficient way to augment IDL with code written in other languages such as C or Fortran. It is, however, not as efficient as writing the required operation entirely in IDL. The actual cost depends primarily on the amount of data being transferred. For example, the above code can be performed entirely in IDL using a simple statement such as the following:
PRINT, 'Average = ', TOTAL(FINDGEN(10))/10.0